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DEVICE, SYSTEM AND METHOD FOR IMAGE BASED SIZE ANALYSIS 

FIELD OF THE INVENTION 

- 5 - - The present invention, relates to a method and system for size analysis 

based on images captured by a camera system. 

BACKGROUND OF THE INVENTION 

10 One of the most important ways a physician has for analyzing a 

pathological condition is to examine the dimensions of the pathological entity. In the 
digestive tract, including the intestines, determination of size of an object within the tract 
can provide important Information useful in diagnosing a condition and prescribing 
treatment. 

15 Prior art systems exist for measuring in vivo distances however, such 

prior art systems suffer from, inter alia, not being able to view or image certain areas of 
the gastrointestinal (Gl) tract, being uncomfortable or difficult to use in some patients. 

Therefore, there is a need for an easy to use size analysis system which 
can more extensively provide views of the Gl tract. 

20 

SUMMARY OF THE INVENTION 

There is provided, in accordance with one embodiment of the present 
invention, a device, system and method for calculating a size of an object using images 
acquired by a moving imager. 
25 In one embodiment there is provided a device, system and method for calculating 

a size of an object in vivo, for example in the Gl tract, using in vivo images acquired by 
a moving imager. In one embodiment, the moving imager includes a single camera. 
The imager may include more than one camera or imaging device. 

There is provided, in accordance with another embodiment of the present 

1 
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invention, a device, system and method for calculation of object size by conversion of 
two-dimensional images, where the two-dimensional images are acquired by a moving 
imager. The embodiment includes a distance-detecting unit for determining a distance 
traveled by the moving imager during the capture of two of the images, and at least one 
5 - processor for generating spatial coordinates, of objects within the images. _ The 
processor uses the distance obtained by the distance-detecting unit, and converts the 
spatial coordinates into a size calculation of the object. 

In one embodiment, the distance-detecting unit uses data provided by a sensor. 
In one embodiment, ^he sensor is a position sensor which has three receivers which 

10 receive signals from a transmitter in communication with the camera system, the 
receiver in communication with a unit for determining the position of the camera 
system. The position sensor may be an induction coil. In another embodiment, the 
sensor is an imager providing data to an image analyzer which can analyze, for 
example, the optical flow of an image. In another embodiment, the sensor is a velocity 

15 sensor, which may be an accelerometer or an ultrasound transducer. 

Certain functions and units, such as a distance detecting unit or a size 
calculation unit, may be completely or partially located externally to the capsule, for 
example in a display station or workstation. 

According to one embodiment, a method for calculating a size of an object in 

20 vivo using images acquired by an imager device includes: determining a distance 

traveled by said imager during capture of two of said images; calculating relative spatial 
coordinates of a set of objects within said images using said distance; and calculating 
the size of one of the set of objects from said spatial coordinates. The distance 
traveled may be non-negligible as compared to a distance between said imager and 

25 said objects. In one embodiment, the moving imager is an in vivo imager. The method 
may include determining the position of the imager, determining the velocity of the 
imager, determining an optical flow of the images, determining the distance traveled by 
analyzing the images, and/or other calculation. The object may be in a gastrointestinal 
tract. 
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The imager may be within, for example, an endoscope, a swallowable capsule, 
or other device. 

In one embodiment a system for calculation of object size using images acquired 
by an imager includes a sensor capable of providing data used for determining a 
5 distance traveled by said imager during the capture of two of. said. images; and. at .Least .. 
one processor for generating spatial coordinates of objects within said images, said 
processor calculating a distance using data obtained by said sensor, wherein said at 
least one processor is capable of converting said spatial coordinates into a size 
calculation of an object. 
10 In one embodiment a system for calculation of object size using images acquired 

by an in-vivo imager device includes: at least one processor capable of receiving data 
from a sensor, said processor capable of generating spatial coordinates of objects 
within said images and calculating a distance using data obtained by said sensor, 
wherein said at least one processor is capable of converting said spatial coordinates 
15 into a size calculation of said object. The sensor may be located in the imager device 
or, alternately, outside a patient's body. The sensor may include an accelerometer, an 
ultrasound transducer, a camera, a position sensor, an induction coil, and/or other 
equipment. The processor may include an image analyzer which is capable of 
analyzing the optical flow of an image and/or which is capable of determining a distance 
20 traveled by analyzing the images. The data obtained by said sensor may be capable of 
being used by said processor to calculate a velocity. 

In one embodiment a swallowable capsule includes an image sensor capable of 
obtaining images from within the gastrointestinal tract; a distance-detecting sensor 
capable of generating data for detemiining a distance traveled by said capsule during 
25 reception of at least two of said images, wherein said data may be used by a processor 
for generating spatial coordinates of at least one object found within said two images 
and for converting said spatial coordinates into a size calculation of said at least one 
object. The sensor may include an accelerometer, an ultrasound transducer, a camera, 
a position sensor, an Induction coll, and/or other equipment. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

The present invention will be understood and appreciated more fully from 
the following detailed description taken in conjunction with the drawings in which: 
. 5 . Fig. 1 is a schematic illustration of a prior art in viyp 

Fig. 2 is schematic illustration of an in vivo camera system transiting part of 
the Gi lumen according to one embodiment of the present invention; 

Fig. 3 is a block diagram illustration of a system according to one 
embodiment of the present invention; 
10 Fig. 4 is a flow chart illustration of the method used by the system shown in 

Fig. 3, according to an embodiment of the invention; and 

Fig. 5 is a schematic illustration showing how spatial coordinates are 
determined according to an embodiment of the present invention. 

Similar elements in the Figures are numbered the same throughout. 

15 

DETAILED DESCRIPTION OF THE INVENTION 

In the following detailed description, numerous specific details are set forth 
in order to provide a thorough understanding of the invention. However it will be 
20 understood by those of ordinary skill in the art that the present invention may be practiced 
without these specific details. In other instances, well-known methods and procedures 
have not been described in detail so as not to obscure the present invention. 

An in vivo video camera system captures and transmits images of, for 
example, the GI tract while the capsule passes through the GI lumen. Some 
25 embodiments may be contained within a capsule; altemate configurations, such as within 
an endoscope, are possible. In addition to the camera system, embodiments contain an 
optical system for imaging an area of interest onto the camera system and a transmitter 
for transmitting image output of the camera. A capsule including such components can, 
for example, pass through the entire digestive tract and operate as an autonomous video 

4 
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endoscope. It may image difficult to reach areas of the small intestine. Embodiments of 
US Patent No. 5,604,531, assigned to the common assignee of the present application 
and incorporated herein by reference, describe an in vivo camera system, which is carried 
by a swallowable capsule. Another in-vivo imaging system is described in International 
5 Application Publication No WO01/65995 published 13 September 2001, assigned to the . . 
common assignee of the present application and incorporated herein by reference. 
While embodiments of the system and method of the present invention may be used 
with devices and methods described in U.S. Patent 5,604,531 and/or International 
Application Publication No WO01/65995, embodiments of the present invention may be 

10 used with other in-vivo imaging systems, having other configurations. 

Reference is made to Fig. 1, which shows a schematic diagram of an in- 
vivo imaging system. The system includes a capsule 40 having, for example, an imager 
46, an illumination source 42, and a transmitter 41 . Outside the patients body are an 
image receiver 12 (for example an antenna array), a storage unit 19, a data processor 14, 

15 an image monitor 18, and a position monitor 16. While Fig.1 shows separate monitors, 
both an image and its position can be presented on a single monitor. 

Imager 46 in capsule 40 may be connected to transmitter 41 also located in 
capsule 40. Transmitter 41 transmits images to image receiver 12, which sends the data 
to, for example, a data processor 14 and a storage unit 19. Data processor 14 may, for 

20 example, analyze the data and may be in communication with storage unit 19, 
transferring frame data to and from storage unit 19. Data processor 14 also may provide 
the analyzed data to image monitor 18 and position monitor 16 where the physician views 
the data. The image monitor may present an image of the Gl lumen and the position 
monitor may present the position in the Gl tract at which the image was taken. The data 

25 can be viewed in real time or at some later date. In addition to revealing pathological 
conditions of the Gl tract, the system can provide information about the location of these 
pathologies. The imaging device may include more than one camera or imaging device. 

Embodiments of the present invention relate to a method, device and 
system of size analysis by converting two-dimensional images, captured by a moving in- 
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Vivo video camera system, such as that of Fig. 1, into three-dimensional representations. 
This conversion is typically done using only one camera or imager, and is typically based 
on the velocity of the camera system when it captures the frames being converted. 

Embodiments of the present invention may be used with embodiments or 

5 modified embodiments of the image capture* reception and display system such as. 
embodiments described in US Patent No. 5,604,531 and/or International Application 
Publication No WO01 65995 published 13 September 2001 . 

Reference is now made to Figs. 2 and 3, which illustrate a video capsule 
40 inside the gut approaching two objects, and a system 15 for determining the size of 

10 one of the objects, according to one embodiment of the present invention. In one 
embodiment, system 15 is located in an external processing and display system, such 
as a personal computer or workstation, including conventional equipment and software 
such as a microprocessor or CPU, a memory, storage, etc. For example, system 15 
may be located In a display system described in embodiments of US Patent No. 

15 5,604,531 and/or International Application Publication No WO01 65995 published 13 
September 2001 . In alternate embodiments, all or part of system 15 or the functionality 
of system 15 may be located in another location or in other equipment. For example, 
some functionality may be in capsule 40. 

In Fig. 2, video capsule 40 is shown approaching a first object 401 and a 

20 second object 402, in Gl lumen 403. Using two, typically, but not necessarily, 
consecutive images captured by capsule 40 and the known speed of capsule 40, size 
analysis based on three dimensional representations of objecte 401 and 402 can be 
done, as will be discussed with regard to Fig. 5 below. 

Referring to Fig. 3, system 15 includes, for example, a distance-detecting 

25 unit 20, an image receiver 12 and a processor 14. Processor 14 includes, for example, 
a spatial coordinate generator 26, a cross correlation 28 and a size generator 30. In 
one embodiment, distance-detecting unit 20 is a position detector. In one embodiment, 
distance-detecting unit 20 obtains a distance measurement d by measuring and 
integrating a velocity, as will be described hereinbelow. Processor 14 may include, for 
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example, a standard PC accelerator board, high performance PC, multiprocfessor PC or 
any other serial or parallel high performance processing machine, and appropriate 
software. Optionally, system 15 may include, for example, an edge detector 22. The 
functionality of one or more of distance-detecting unit 20, spatial coordinate generator 
-5 • 26,- cross correlator 28, edge detector 22 and/or size generator .3Q may. be cQntained 
within or controlled by processor 14. Any suitable edge detector or edge detection 
capability used in conventional image analysis can be used, such edge detection 
capability using the following sliding window filter: 

Reference Is now made to Fig. 4 which is a flow chart diagram Illustrating a 

■-1 0 -r 

0 4 0 
-1 0 -1_ 

10 general method for generating size measurements from two-dimensional images 
according to one embodiment of the present invention. Steps of Fig. 4 may be 
accomplished using, for example, the system 15 of Fig. 3. In another embodiment, the 
embodiments of the methods described herein may be practiced using equipment or 
devices of other configurations. First, imager 46 within a moving in vivo video camera 

15 system such as the one described in Fig. 1) captures (step 101) images periodically, such 
as every 100-1000 ms. In one embodiment, the images are captured every 500 ms. 
Image data is transmitted to image receiver 12 (Fig. 1). Data processor 14 divides 
received images into a grid of pixels, and selects (step 102) pixels for analysis. As in 
other imaging applications, the number of pixels determines the resolution of the image. 

20 For purposes of this discussion, the images are divided into m x n pixels. 

In one embodiment, a user may choose which object or objects are to be 
tracked for size analysis. For example, a user may select an object, objects, or region 
being displayed in a moving or still image on a monitor using, for example, a pointing 
device such as a mouse. According to one embodiment an object is selected in a manner 

25 similar to selecting an object during ultrasound procedures. In another embodiment the 
system may automatically chose an object(s) or region for size analysis. In one 
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embodiment, the calculations are performed on a workstation with stored data, and a user 
is able to move forward and backwards through a moving image and see or request size 
information for the various objects displayed during the portion viewed. In another 
embodiment, size analysis may be performed in real time. Size information may be. for 
5 example, stored and/or displayed.- Other object or region selecting methods may be 
used. 

Next, cross correlator 28 calculates (step 104) an xy cross correlation 
function between the intensities Ij and Ij+n of image j and image j+n, thereby identifying 
corresponding pixels in images j and j+n. The value n is typically, but not necessarily, 1. 

10 Henceforth, the second frame will be designated as j+1 , with the understanding that n can 
also be greater than 1 . 

The correlation can be done for each of the m x n pixels created in images 
j, and j+1. However, in another embodiment, edge detector 22 selects (step 106) pixels 
for cross correlation, thereby selecting an object. In one embodiment, only pixels whose 

15 edges exceed a certain predetermined threshold value are selected for correlation. Other 
suitable methods for image processing may be used. 

While the cross correlation can be done on a pixel by pixel basis, more 
typically, it is performed on parts of the image, such as sets of 8x8 pixels. The latter 
approach can be used to minimize computation time. 

20 In one typical cross correlation function, the cross correlation coefficient Cxy 

is given by: . 

Cxy = E E l](m,n)lj+i(m+x, n+y) 

m n 

25 

where Ij(m.n) and lj+i(m,n) are the intensity values of pixel (m.n) in images j and j+1 
respectively. The vector (x, y) can be considered the displacennent vector from pixel 
(m,n) in going from pixel (m,n) to pixel (m+x, n+y). The maximum of the cross con-elation 
function indicates the most probable location of con^espondence between the pixels of 
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images j and j+1 . A suitable cross correlation function is Included in Matlab. a standard 
mathematics package for computers; other functions may be used. 

The results of the cross correlation may provide, for example, x and y 
coordinates for a specific point If the cross correlation is performed for, for example, four 
5 edges of an object on images j and an entire two-dimensional set of spatial 
coordinates is obtained (step 108). Thus, for object A, xia, X2a, yiA and y2A are known. 

The determination of the z coordinates for object A is typically based on 
the distance traversed by imager 46 while it moves through the Gl tract capturing 
images j and j+1. Typically, distance traveled is determined from data captured by a 

10 sensor which is typically located within the device containing the capsule 40; the sensor 
may alternately be external to the patient. In one embodiment, distance-measuring unit 
20 measures the velocity of imager 46 using an accelerometer and an integrator. 

The accelerometer may be, for example, the ADXL50 model from Analog 
Devices. It is readily evident that, in addition to an accelerometer, any sensor that can 

15 provide data to determine the velocity of the capsule could also be used. Such sensors 
include, but are not limited to, induction coils (as described in U.S. Patent Number 
4,431,005, incorporated herein by reference) and ultrasound transducers. For example, 
if an induction coil is located in the capsule and the patient is placed in a magnetic field, a 
current would be produced by the coil with a magnitude proportional to the velocity of the 

20 capsule. Similarly, ultrasound transducers, such as those used in conventional medical 
ultrasound devices, can be used as an extemal sensor to track the movement of the 
capsule and standard electronics could be used to convert the data to velocities. Other 
distance measurement systems and methods may be used. 

In another embodiment, the change of position of the capsule while 

25 capturing two images can be used to determine the distance traveled by the capsule 
during the time interval between the images. Signals sent by a transmitter within the 
capsule and received by receivers outside the body can be used to locate the position of 
the capsule. One suitable system for determining capsule location is one described in 
Published U.S. Application Number US-2002-0173718-A1 assigned to the common 
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assignee of the present application and incorporated herein by reference; other suitable 
location determining systems may be used. 

In yet another embodiment, conventional imiage analysis techniques can be 
used to analyze the optical flow of the images. For example, on the basis of the smear 

5 - pattern of the images, velocity or distance can be determined. Once the velocity is 
known, a, for example, integrator calculates (step 1 1 2) the distance traveled by imager 46 
from the time of capture of image j to the time of capture of image j+1. This distance 
value is used in determining (step 116) the z coordinate of object A, as described in the 
methods provided as examples herelnbelow. Other methods may be used within the 

10 scope of the present invention. 

One method described hereinbelow is adapted from a method discussed in 
Manhir^e^ Visrnn- Theory , Algorithms, Practicalities , E.R. Davies, Academic Press 1996, 
pp. 441-444, incorporated herein by reference. Davies describes how a camera, when 
moving along a baseline, sees a succession of images. Depth information can be 

15 obtained by analyzing the object features of two images. 

In general, the discussion by Davies uses far-field approximations; he 
discusses systems where the distance traveled by a camera between images is far 
smaller than the distance to the object. That is a condition that typically does not apply to 
in vivo video camera systems imaging the GI tract. In vivo video camera systems typically 

20 move distances that are non-negligible in size when compared to the distances between 
the camera and objects being imaged. Because far field approximations are typically not 
valid for in vivo video camera systems. Images of two objects may be required, where one 
object serves as a reference object. 

Reference is now made to Fig. 5, which shows a geometric illustration of 

25 the basis for calculating the z coordinate of a set of objects A and B (where set can 
include one item), according to one embodiment of the present invention. It should be 
noted that the z coordinate represents the distance from imager 46 to each of the objects, 
denoted ^ and zb respectively. 

As mentioned above. Imager 46 typically moves a certain distance d from 

10 
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the capture of the first image 202 to the capture of the second image 204 (of course 
periods of little or no movement are possible). Thus, the distance between images 202 
and 204 is distance d. In addition, there is a certain focal length f, which is the lens focal 
length. While in one embodiment focal length f is used in the derivation of the following 

• 5- equations, it is typically eventually eliminated and in such a case its value does not need 
to be known explicitly. 

The projections of objects A and B on each of the images 202 and 204 in 
the y direction are shown in Fig. 5 and are denoted a1, b1, a2 and b2, respectively. 
These values are obtained from, for example, the pixel information stored in storage unit 

10 19, and correspond to the n value of each m x n pixel. Thus, for example, a1 represents 
the X value of object A as it was acquired in time t1 (Xia) and a2 represents the X value of 
object A as it was acquired in time t2 (X2a). Accordingly, b1 represents the X value of 
object B as it was acquired in time t1 (Xib) and b2 represents the X value of object B as it 
was acquired in time t2 (X2b). 

15 The actual values for a1 , a2. b1 , and b2 may be calculated by, for example, 

image processor 14 (step 108 of Fig. 4) from, for example, the size of the image sensor in 
imager 46 and image pixel data stored in storage unit 18. Thus, if the image sensor has 
a length L, and there are m pixels along the X axis, then an object whose length is p 
pixels will have an actual size of: L * P/m. 

20 Using similar triangles, it can be shown that the following relationship exists; 

Zb(1-Tb) = Za(1-Ta) + d(Tb-Ta) 

where Ta and Tb are defined as: 
25 . Ta = ai/a2 

Tb = bi/b2 

Thus, the z coordinate for object A as a function of the z coordinate for 

11 



wo 03/053241 



PCT/IL02/01025 



Object B can be obtained. Spatial coordinate processor 26 calculates (step 116) the z 
values for two points on object A (zia and Z2a) con-espondlng to the two edges of object A. 
Accordingly, xyz spatial coordinates are known for object A. Size analyzer 30 then 
calculates (step 118) the size of object A by, for example, subtracting each of the axis 

. 5 . coordinates fronn. each other.. . Thus, Xf, = Xz^^-Xia; Va = ygA-yiAi and za = ^-ziA,.resultirig in 
values for length, width and height, respectively, of object A. 

Other fomnulas and methods of calculation can be used, falling within the 
scope of the present invention. 

Alternatively, other methods can be used to calculate Za, such as one 

10 based on the follovAnng exemplary relationships: 

A/a-i = (Za + d + f)/f 
A/a2 = (Za + f)/f 

15 

From those two equations the following can be calculated: 
A*f=(ZA + d + f)yai = (Za + O* 32 

20 Leading to 

ZA*ai-ZA*a2 = f*a2~(d + f)*ai = f *(a2-ai)-d *ai 

Finally, 

25 

Za= d*ai /(a2-ai)-f 

Thus, if the focal length of the camera is known, only one object is needed for calculation. 

12 
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The size of the object may calculated, for example, as described above. 

Image processor 14 may send any selected size-data to image monitor 18 
for display. Such size data may be displayed along with, for example, image or moving 
image data on Image monitor 18. Various methods of displaying the size data may be 
5 used. - 

The procedure described hereinabove can be performed as a post- 
processing step, or, with adequate computational capability, it can be done in real time, 
allowing the user to choose specific images for processing. 

It should be evident that while Fig. 5 shows a one-dimensional object, (e.g. 
10 a line), here positioned along the X-axis, symmetry considerations can be used in an 
analogous manner to obtain the Y coordinate, where the Y-axis is perpendicular to the 
plane of the paper. 

For the above described methods, other steps or series of steps may be 
used. Furthermore, with the various formulas and methods of calculation presented 
15 above, other formulas and methods of calculation can be used, falling within the scope of 
the present invention. While specific formulas and dimensions are presented, other 
formulas, or variants of the formulas presented, and other dimensions, may be used. 

It will be appreciated by persons skilled in the art that the present 
invention is not limited to what has been particularly shown and described hereinabove. 
20 Rather the scope of the present invention is defined only by the claims that follow: 
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What is claimed is: 

1 . A method for calculating a size of an object in vivo using images acquired by an 
5 imager, the method comprising: 

determining a distance traveled by said imager during capture of two of said 
images; 

calculating relative spatial coordinates of a set of objects within said images 
using said distance; and 
1 0 calculating the size of one of the set of objects from said spatial coordinates. 

2. The method according to claim 1 wherein said distance traveled is non-negligible 
as compared to a distance between said imager and said objects. 

]5 3. The method according to claim 1 wherein the imager is included within an 
endoscope. 

4. The method according to claim 1 wherein the imager is included within a 
swallowable capsule. 

20 

5. The method according to claim 1 comprising determining the position of the 
imager. 

6. The method according to claim 1 comprising determining the velocity of the 
25 imager. 

7. The method according to claim 1 comprising determining an optical flow of the 
images. 

14 
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8. The method according to claim 1 comprising determining the distance traveled 
by analyzing the images. 

9. The method according to claim 1 wherein the object is in a gastrointestinal tract. 



10. A system for calculation of object size using images acquired by an imager, said 
system comprising: 

a sensor capable of providing data used for determining a distance traveled by 
said imager during the capture of two of said images; and 
10 at least one processor for generating spatial coordinates of objects within said 

images, said processor calculating a distance using data obtained by said sensor, 
wherein said at least one processor is capable of converting said spatial coordinates 
into a size calculation of an object. 

15 11. The system of claim 10, wherein said sensor is located in a swallowable 
capsule. 

12. A system for calculation of object size using images, said images acquired by an 
in-vivo imager device, said system comprising: 
20 at least one processor capable of receiving data from a sensor, said processor 

capable of generating spatial coordinates of objects within said images and calculating 
a distance using data obtained by said sensor, wherein said at least one processor is 
capable of converting said spatial coordinates into a size calculation of said object. 

25 13. The system of claim 12 wherein the sensor is located in the imager 

device. 

14. The system of claim 12 wherein the sensor is located outside a patient's 

body. 

15 
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1 5. The system according to claim 12 wlierein said sensor Includes a camera. 

16. The system according to claim 12 wherein said sensor includes a position 

. 5 . sensor . . , ... 

17. The system according to claim 12 where said sensor includes an induction coil. 

18. The system according to claim 12 wherein said processor includes an image 
10 analyzer which is capable of analyzing the optical flow of an image. 

19. The system according to claim 12 wherein said processor includes an image 
analyzer which is capable of determining a distance traveled by analyzing the images. 

15 20. The system according to claim 12 wherein said data obtained by said sensor is 
capable of being used by said processor to calculate a velocity. 

21 . The system according to claim 20 wherein said sensor includes an 
accelerometer. 

20 

22. The system according to claim 20 where said sensor includes an ultrasound 
transducer. 

23. The system according to claim 12 wherein the imager device is disposed within 
25 an endoscope. 

24. The system according to claim 12 wherein the imager device is disposed within a 
swailowable capsule. 

16 
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25. A swaltowable capsule comprising: 

an innage sensor capable of obtaining images from within the gastrointestinal 

tract; 

a distance-detecting sensor capable of generating data for determining a 
distance traveled by said capsule during reception of at least two of said images, 
wherein said data may be used by a processor for generating spatial coordinates of at 
least one object found within said two images and for converting said spatial 
coordinates into a size calculation of said at least one object. 

26. The capsule according to claim 25 where said position sensor includes an 
induction coil. 

27. The capsule according to claim 25 wherein said sensor includes a velocity 
sensor. 

28. The capsule according to claim 25 wherein said sensor includes an imager. 
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